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In this paper we report the interview questions used in a study of middle school students’ 
mathematical problem solving behaviors which were chosen based on Vygotsky’s concept formation 
theory and Berger’s appropriation theory. We discuss the task design/selection process along with 
the findings associated with the use of these tasks so to provide new direction for gauging research 
on mathematical problem solving. 
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Introduction 

Improving the teaching and learning of mathematical problem solving relies heavily on 
development of a theory of mathematical problem solving, which is currently missing from the field 
(Schoenfeld, 2007). As Schoenfeld articulated, the focus of mathematics education researchers 
interested in mathematical cognition may need to shift on building theoretical capacity that may 
account for human decision making in the course of problem solving (2013). We posit that progress 
towards building such a theory might demand greater attention to theorizing instrument design and 
task development in conducting research on mathematical problem solving, an area currently absent 
from the field. Discussions surrounding tasks have frequently focused on defining problems: a 
question where an individual does not have a ready- to-use approach to find the answer (Wilson, 
Fernandez, & Hadaway, 1993). Agreement exists that whether a question is a problem depends on 
the individual working on the task (Schoenfeld, 1985). This description has imposed constraints on 
researchers’ ability to theorize specific principles of instrument design/selection when undertaking 
research on mathematical problem solving. Existing studies have generally selected questions based 
on the targeted subject area and whether the question is appropriate for the participants in the study, 
referring to the participants’ educational background as a standard to determine whether a question is 
beyond their capability or not (e.g. Elia et al., 2009; Kuzle, 2011). These efforts, although useful in 
providing a profile of expert problem solvers, do not provide a coherent perspective on 
instrumentation as a methodology. 

In a larger study of middle school students’ mathematical problem solving behaviors, we used 
research-based assessment to capture students’ ways of knowing by unpacking the relationship 
among mathematical concepts, cognitive behaviors, and metacognitive behaviors as evidenced 
during clinical interviews. We reported the findings from the pretest instrument in the study, which 
suggested that Vygotsky’s concept formation theory could serve as an effective framework for 
designing novel assessments to provide researchers with more precise tools to articulate intricacies 
of students’ understanding of mathematical concepts (Zhang, Manouchehri, & Tague, 2013). In 
this paper we will report on theoretically based criteria for design of tasks to be used in our 
research on mathematical problem solving. We will report findings related middle school students’ 
performance on a selected sample of these tasks so to provide direction for future research on 
mathematical problem solving. 


Theoretical framework 
As a Starting point in our task design, two issues were of particular concern: (1) establishing 
theoretical capacity that would allow us to document and analyze both cognitive and metacognitive 
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behaviors of the participants in the course of their problem solving; (2) selecting a specific 
mathematical concept so to develop appropriate tasks surrounding it. To address the first issue we 
incorporated two theoretical perspectives to design/select interview questions used in this study: 
Vygotsky’s (1962) concept formation theory and Berger’s (2004) appropriation theory. Vygotsky’s 
theory proposes a framework for an individual’s concept development within a social environment, 
while Berger’s theory proposes an interpretation of Vygotsky’s theory in the domain of mathematics 
by adjusting certain stages. Both theories break down any concept development into three phases: 
heap, complex, and concept. In the heap phase, the learner associates a sign with another because of 
physical context or circumstance instead of any inherent or mathematical property of the signs. In the 
complex phase, objects are united in an individual’s mind not only by his or her impressions, but also 
by concrete and factual bonds between them. In the concept phase, the bonds between objects are 
abstract and logical. For the purpose of the study we also decided to focus on the concept of Area 
due to its critical role in school mathematics. Hence, research-based formation stages for the concept 
of area were assembled as illustrated in Figure 1. 


1.1 Non-M: Connection between the child’s impression 


Heap 
1.2 M: Unreasonable estimate 
2.1.1.1 Non-M: compare parts randomly 
Surface 2.1.1.2 Unit area: iterate incorrectly 
2.1.1.3 Formula: incorrect 
Association 1 2.1.2.1 Unit area: correct iteration of wrong unit 
Example-oriented [ correct iteration of whole but not factional units 
2.1.2.2 Formula: only used under specific occasions 
Artificial — 2.1.3 M: determine perimeter; count dots in a shape 
Concent 2.2.1 Non-M: use third object to compare whole area 
EP iee Complex Chain al 


(connection) 2.2.2 M: use a third object to compare area 


Representation —— 2.3 Non-M: generalize properties/conclusions of objects based on representations 


2.4.1 Non-M: compare parts non-rigorously 
2.4.2 M: empirical estimate 
Pseudo-concept 


2.4.3 Unit area: Non-rigorous estimations on visible area units 
2.4.4 Formula: correct fomula with computational errors contradicting to 
other reasoning but unable to justify 
3.1.1 Unit area: correct operation on visible area units 


Potential concept [ 


3.1.2 Formula: understand procedures and formulas 


Concept 
(abstraction) 3.2.1 Non-M: compare parts systematically 
Concept E Unit area: correct operation on invisible area units 


3.2.3 Formula: generalized 


Figure 1. Developmental Stages of the Concept of Area 


This structural model served as our primary analytical tool for qualifying the students’ work as 
well as the tasks selected to be used in study. This model was further refined upon a short pilot study 
in which responses to tasks were obtained from 44 middle school students. The students were asked 
to responds to five items. Upon analysis of their responses we considered revisions of some of the 
tasks so to assure ambiguities that could lead to irrelevant answers were removed. Analysis of the 
relationship among mathematical concepts, cognitive and metacognitive behaviors emerged from the 
interview results based on our final selection of items which were used in in-depth interviews with 
five middle school students. The current report is based on our findings of these interviews. 
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Methods 


Participants 

Five individuals from a population of 44 sixth grade students were selected to participate in 
interviews. The original group of 44 were enrolled in three distinct class periods of an algebra course 
taught by the same teacher. All three classes had been observed by the lead author for 6 months prior 
to data collection. A pretest was administered to the 44 students. The process of determining and 
classifying the pretest responses is described as the following. 1) Two authors independently 
reviewed all responses to identify and document enacted approaches and coded developmental stages 
associated with each approach based on an earlier version of the developmental stages framework. 
Notes were compared for consistency in coding. 2) Students’ approaches that were ambiguous or 
non-anticipated were discussed. The framework was adjusted based on the analysis of these 
responses; five more stages were identified and added to the original framework. 3) Based on the 
analysis we built a reference list of detected stages for each student to inform participant selection. 
Each individual’s developmental status revealed in the responses was categorized as “overall low” 
(all responses were rated as Heap and non-Pseudo- concept Complex stages), “varied” (responses 
were rated across Heap to Concept stages), and “overall high” (Reponses were rated as Pseudo- 
concept Complex and Concept stages). 

Participant selection was deliberate. The following criteria guided our choices: 1) willingness to 
be involved in the study and had signed the consent forms; 2) the participants would need to 
represent a range of different developmental attributes pertaining to the target concept. This data was 
collected through the students’ pretest responses; 3) the participants would need to be comfortable 
with thinking aloud. Five individuals met these criteria: Shana exhibited a /ow status, Andy exhibited 
a high status, Sandy, Allen, and Ivan exhibited varied status. 


Data collection 

The five participants were interviewed individually. Each interview consisted of two parts: 
During the background interview part the participants’ mathematics background information, their 
beliefs about mathematics, and their views on the value of mathematics for their lives were elicited. 
The second part, problem solving session the participants worked on specific mathematical tasks. 
During the problem solving sessions, interviewer interventions were limited to eliciting clarifications, 
explanations, or justifications, when needed. 


Instrument design 

F ive problems were used during the interviews. All problems were related to the concept of area 
and allowed the participants to tackle the tasks from different stages of concept development. The 
problems could potentially cover a wide range of concept stages. Peripheral concepts were integrated 
in several problems in order to enable some degree of interactions between different concepts (e.g. 
transformational reasoning and variable), since we believe authentic problem solving should involve 
more than one concept yet the number of concepts needs to be controlled for a deeper analysis on the 
behaviors acting upon them. When selecting tasks, concept stages along with the corresponding 
exemplar approaches that could be elicited by each problem were predicted by the authors. An 
example of this conceptual mapping is shown in Table 1. 
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Table 1. Predicted stages and exemplar approaches 


Compare areas problem 
Which of the regions shown below has the largest area? How would you order them? 


Predicted stages Exemplar approach 

Surface Association Complex — Formula Using an incorrect area formula 

Chain Complex Fitting the circle and the triangle into the 
square 

Potential Concept - Formula Using a correct area formula 


Data analysis 

Data collection consisted of four phases. First, each participant’s key cognitive behaviors during 
each problem solving episode were mapped and documents. Second, a summary of observed concept 
stages and metacognitive behaviors during the episode were catalogued and noted. Detailed analysis 
of the individual’s problem solving behaviors according to the relationship among concepts, 
cognitive behaviors, and metacognitive behaviors was constructed. Finally, cross analysis of the 
observed concept stages, metacognitive behaviors, and the relationship between them concluded the 
analysis phase. This process was followed for each of the five tasks used. 


Results 

The larger study analyzed the participants’ performances on each interview question from four 
perspectives: 1) each participant’s point of entry, including identified task elements/objectives and 
his/her initial approach, 2) types of approaches the study participants used, 3) concept stages revealed 
when working on the problem, and 4) metacognitive behaviors revealed from the participants during 
the problem solving episode. 

To focus on the research question in this paper, we only report on: 1) each participant’s point of 
entry, including identified task elements/objectives and his/her initial approach, 2) types of approaches 
the participants used, and 3) concept stages revealed. The problem in Table 1, Compare Areas 
problem serves as an illustrative example of research-based tasks used in the larger study reported 
elsewhere (Zhang, 2014). 


Point of entry 

In the Compare Areas problem, one task element and one task objective are essential to solving 
this problem: the variable a that represents the equal length of sides and diameter, and the 
comparison of the areas, respectively. 

Table 2 summarizes the task elements/objectives (i.e. key conditions and goals to solve the 
question) identified by each participant and by the interviewer before each of the participants started 
solving the problem. Initial approaches adopted by each participant are also outlined. 
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Table 2. Summarization of initial response/approach adopted by each participant 


Participant | Task elements or Task elements or Initial approach 
objectives objectives 
identified by the highlighted by the 
participant interviewer 
Shana Equal measure of NA The circle and the triangle 
sides and diameter could fit into the square. 
eae a ai. 
ee 
\ 
Sandy Just area The a meant equal Computed the areas by 
measure of sides and | formulas. 
diameter 
Ivan NA The circle could fit into the 
square (verbal conclusion); 
the triangle had the same area 
as the circle since the three 
vertices of the triangle “pokes 
out.” 
Andy Around is the The a meant equal Computed the areas of the 
circumference. measure of sides and | square and the triangle by 
diameter formulas. 
Allen The area inside NA The circle and the triangle 
could fit into the square. 


Among the five participants, Sandy’s understanding of variable greatly influenced her initial 


behaviors. She focused on the actual amount of area instead of the condition that the lengths of the 


sides and the diameter had the same measures. This prevented her from adopting either a visual or a 
numerical approach to start tackling the problem. Allen, whose understanding of variable was 


restricted, chose to ignore it so to avoid confusion. Ivan overlooked the variable; his later behaviors 
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were influenced by this element rather than at the beginning. Interestingly, Shana (who was assessed 
as low concept development) was the only participant who explicitly identified the task element of 
equal measure, whereas the others were either confused by the information, overlooking it, or needed 
to be reminded by the interviewer. 


Documented approaches 
Table 3 illustrates the approaches that the participants used when solving the Compare Areas 
problem. 


Table 3. Approaches adopted by the participants solving the Compare Areas problem 
Approach Description Example from interview 
Fit into the Showed the circle and 
square the triangle could fit into 

the square. 


Formula Used area formulas to Assigned a number to a and used the 
compute the areas. measure of the side of the triangle as 
its heigh 
Unit squares Drew and count unit 
squares to approximate 
the area. 


Transformed the 
leftovers of the circle 
and the triangle into 
manageable shapes for 
comparison. 


Compare leftover 


The third approach was adopted when a specific area formula (i.e. the area formula of circles) 
was not available to Andy who used the second approach as his initial heuristic. The fourth approach 
was usually adopted when the participants, who used the fit-into-the-square approach as their initial 
approach, were prompted to further compare the areas of circle and triangle. 


Revealed concept stages 

When designing the interview instrument, the researchers had predicted six concept stages that 
could be revealed in this problem: 1) Surface Association Complex — Formula wherein a student uses 
an incorrect formula, 2) Artificial Association Complex where perimeter is compared instead of area, 
3) Chain Complex - Non-measurement wherein the circle and the triangle can fit into the square, 4) 
Pseudo-concept Complex - Formula where visual reasoning contradicts to computational answers, 5) 
Potential Concept - Formula where a number is plugged into a formula, and 6) Concept — Formula 
where variable a is plugged into the formula to reach a generalized answer. Among these stages, 1, 3, 
and 5 were revealed during the interviews, while 2 was observed during the identification of task 
objectives prior to one participant’s initial approach. 

Table 4 summarizes the concept stages of Area concept revealed from the five participants when 
solving this problem. The approaches associated with each stage as used by them are noted as well. 
The stages listed in italic are the ones predicted to be revealed by the researcher. 
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Table 4. Concept stages of area and associated approach revealed by participants 
Revealed concept stage Associated approach 


Surface Association Complex — Non-measurement Compare leftover 
Surface Association Complex — Formula Formula 


Fit into the square 
Compare leftover 
Unit squares 
Formula 


Concept - Non-measurement Compare leftover 


All other stages were observed when the participants sought an alternative way to either refine or 
complement their initial approaches. The participants’ problem solving behaviors were much more 
novel during this process (thus less predictable) than their behaviors during the initial attempt (i.e. 
using fit-into-the-square or formula approach). 


Discussion 

The study results suggested that metacognitive behaviors of the participants are closely linked to 
their perceived level of complexity of the task and whether they found it appropriately challenging. 
That was the case regardless of the participants’ level of development pertaining to the concept under 
study (in our case, area, for instance) or their personal preferences for particular strategies. That is, 
when a task was perceived as too challenging, it was treated as enigma. This limited our capacity as 
researchers to access cognitive or metacognitive behaviors of the participants. The same occurred when 
the problem was perceived as trivial. Because of this, we argue that description of what constitutes a 
problem (in the context or problem solving) needs to be more precisely defined. The common 
description “whether a task is a problem depends on the individual” (Schoenfeld, 1985) too broad to 
inform research instrumentation when studying mathematical problem solving. We posit that 
benchmarks for selection/design/ development of appropriate problems, according to the concept 
formation stages specific to those embedded in the task, can enable researchers to more adequately 
elicit student problem solving. This issue is particularly critical if inferences are to drawn regarding the 
individuals’ understanding of a concept in presence of their mathematical problem solving 
performance. 

Since existing literature does not provide a guide that can inform task design/selection for 
research purposes, focused scholarly efforts towards construction of such a theoretical platform are 
needed. Towards that goal several key issues merit elaboration, among many include: 1) task 
elements and objectives that are essential to understanding the problem in light of learning 
progression, 2) specific concept stages as entry points to the tasks, 3) desired concept stages 
embedded in solutions, and 4) potential shifts/paths between concept and its associated concept 
stages. 

Discussion of task elements need to identify conditions under which a problem may suit different 
populations according to their experiences with the concepts involved. Naturally, if the problem solver 
has had limited or no experience with a particular concept their interpretation of the task or what is 
expected as an appropriate answer may not match those of the researchers. Discussion of specific 
concept stages as entry points granted by the tasks allows the researcher to use the same task with 
different populations who may not share the same concept developmental status. Such an elaboration 
allows us to determine whether the task might be too challenging/impossible to solve given a specific 
concept developmental stage. A description of potential shifts and paths can serve as an aid for the 
interviewers/researchers to gauge their interventions during the interviews (potential prompts/probing 
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questions when observing problem solving process). These descriptions mainly determine the capacity 
of a task. If a task allows only one entry point and one desired stage for the solution, it mostly assesses 
whether an individual knows the procedure or not, which is commonly used in proficiency tests. The 
development a detailed theoretical model of task design is a necessary tool towards development of a 
theory of mathematical problem solving. 
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